Home | | Latest | About | Random
# 27 Inverse of a matrix.
We have been looking at linear systems of the form $Ax=b$, and you may wonder, why can't we just "divide" $A$ over to the right hand side, and write something like $x=\frac{b}{A}$ or $x = A^{-1}b$. This isn't totally unreasonable, but as we will see this is not always possible. A quick reason why to see why this isn't always possible is because if so, then the formula $x=A^{-1}b$ suggests one unique answer $x$ to $Ax=b$, which we know a linear system need not always give unique solution. In fact, a linear system is not always consistent either. So this already puts some limitation of when we can do this. The idea is **invertibility**, and we will investigate when this is possible for a matrix.
## Identity matrix.
Recall we write $I_{n\times n}$ or $I_{n}$ to be the $n\times n$ **identity matrix**, and they look like a diagonal of 1's: $$
I_{n\times n}= I_{n} = \begin{pmatrix}1 & & & 0\\
& 1 &\\
& & \ddots \\
0& & & 1\end{pmatrix}
$$This is called the identity matrix because whatever appropriately sized matrix or vector $I_{n}$ multiplies to, or gets multiplied by, would not change. That is
> If $A$ is of size $n\times k$, then $$
AI_{k\times k} = A \quad\text{and}\quad I_{n\times n}A=A.
$$
Notice the sizes of matrices still matter when you multiply matrices.
## Invertibility.
Given a **square** matrix $A$ of size $n\times n$, we define the following
> **Definition.** We say a square $n\times n$ matrix $A$ is **invertible** if there is another matrix $B$ such that both $$
AB = I_{n\times n} \quad\text{and}\quad BA=I_{n\times n}.
$$In this case, we denote $B$ to be $A^{-1}$ and say it is **the** inverse of $A$.
If a square matrix $A$ is not invertible, then we call it **singular**. And we also say an invertible square matrix as **non-singular**.
Here $I_{n\times n}$ is the $n \times n$ identity matrix. Note the two-sidedness is important in the definition of invertibility.
Also in the definition, we use the phrase "$A^{-1}$ is the inverse of $A$" when $A$ is invertible, this suggests that this inverse, when exists, is unique. Let us prove it.
> **Proposition.** If $A$ is invertible, then its inverse is unique.
$\blacktriangleright$ Proof. Say $A$ is of size $n \times n$. If $A$ is invertible, this means there is a matrix $B$ such that $$
AB = BA = I_{n\times n}.
$$This $B$ is an inverse of $A$. If there is another matrix that also serve as the inverse of $A$, say $\tilde B$, then we also have $$
A\tilde B = \tilde BA = I_{n\times n}.
$$We like to show that in fact $B=\tilde B$, hence establishing the inverse of $A$ is in fact unique. To do this, we note that $$
BA = \tilde B A,
$$since both quantities equal to the same identity matrix. Apply $B$ on the right on both sides of the equation, we get $$
\begin{array}{rrcll}
& BAB & = &\tilde BAB \\
\implies & B(AB) & =&\tilde B(AB) & \text{, by associativity} \\
\implies & BI & =&\tilde BI & \text{, since \(AB=I\)} \\
\implies & B & = & \tilde B & \text{, since \(I\) is identity matrix.}
\end{array}
$$Hence we see that $B = \tilde B$, and that the inverse to $A$ is indeed unique. $\blacksquare$
Again, we often just write this inverse matrix as $A^{-1}$ when $A$ is invertible, and in this case $AA^{=1}=I=A^{-1}A$.
**Example.**
Verify that the matrix $A=\begin{pmatrix}1&2\\3&4\end{pmatrix}$ has an inverse matrix $A^{-1}=\begin{pmatrix}-2&1\\3 / 2&-1 / 2\end{pmatrix}$.
$\blacktriangleright$ Indeed, note $$
AA^{-1}=\begin{pmatrix}1 & 2 \\
3 & 4\end{pmatrix}\begin{pmatrix}-2 & 1 \\
3 /2 & -1 /2\end{pmatrix}= \begin{pmatrix}1 & 0\\0 & 1\end{pmatrix}
$$and $$
A^{-1}A = \begin{pmatrix}-2 & 1 \\
3/2 & -1/2 \end{pmatrix} \begin{pmatrix}1 & 2 \\
3 & 4\end{pmatrix}=\begin{pmatrix}1 & 0 \\
0 & 1\end{pmatrix}. \quad\blacklozenge
$$
**Example.**
Check that the identity matrix $I_{n}$ is invertible, whose inverse is itself. And that the zero matrix $O_{n\times n}$ is not invertible. $\blacklozenge$
Notice in the proof of the uniqueness of inverses, we used the fact that $A$ is invertible to "cancel" $A$. But what is happening is that we have cancellation law because we can multiply by the inverse of $A$. That is,
> **Cancellation laws.** Assuming $A$ is an invertible matrix. Then $$MA=NA \implies M=N$$Also, if $A$ is invertible and whenever $AX=AY$, we have $X=Y$.
$\blacktriangleright$ Proof. We will prove one of them and leave the other one as exercise.
Suppose $A$ is invertible and that $MA=NA$, then multiply $A^{-1}$ (which exists as $A$ invertible) on the right on both sides, we get $$
\begin{align*}
MAA^{-1} & = NAA^{-1} \\
MI &= NI \\
M & = N,
\end{align*}
$$as claimed. I leave the other statement as exercise. $\blacksquare$
**Caution.** Typically we do not have matrix cancellation laws! That is in general, $AB=AC$ does not imply $B=C$!
Now, it is important to verify that this proposed inverse matrix is a two-sided inverse, in the sense that when multiplied on the left **and** on the right to the original matrix, we produce identity matrix in both cases.
However we can rest assured that once we found a one-sided inverse for a square matrix, it is automatically an inverse on the other side as well:
> **Theorem. (Right-invertible iff left-invertible for square matrices.)**
> Suppose $A$ and $B$ are $n\times n$ matrices such that $$
AB=I_{n\times n}.
$$Then we also have $$
BA = I_{n\times n}.
$$
Though this looks "simple" but it depends on the finiteness of the size of the matrix to prove it. In general this is false for general linear maps. Let us take it as fact for now.
So, if we can just find a right inverse to a square matrix $A$, we have found an inverse. Let us discover some procedures in finding these inverses, if exist.
## Methods of finding matrix inverse.
For $2\times 2$ matrices $A$, we can actually write down a simple formula:
> **Proposition.** Let $A=\begin{pmatrix}a&b\\c&d\end{pmatrix}$. Then it is invertible if and only if the quantity $ad-bc \neq 0$. In this case, its inverse is given by $$
A^{-1}= \frac{1}{ad-bc} \begin{pmatrix}d & -b\\-c & a\end{pmatrix}
$$
**Remark.** The quantity $ad-bc$ is actually something called the **determinant** of $A$, denote as $\det(A)$. And the matrix $\begin{pmatrix}d&-b\\-c&a\end{pmatrix}$ is actually called the **adjugate** of the $2\times 2$ matrix $A$, denote as $\text{adj}(A)$. In general, if a matrix $A$ is invertible, it does have a formula of the form $A^{-1}=\frac{1}{\det(A)}\text{adj}(A)$, but this requires a discussion on determinant and adjugate matrix. We will visit this later (at least the determinant).
**Example.**
Is $A = \begin{pmatrix}1&1\\2&2\end{pmatrix}$ invertible? $\blacktriangleright$ If we believe the proposition, note that $\det(A)=2-2=0$, hence $A$ is **singular** and **not invertible**. $\blacklozenge$
**Example**
Is $A = \begin{pmatrix}0&2\\4&8\end{pmatrix}$ invertible? $\blacktriangleright$ If we believe the proposition, since $\det(A)=0-8=-8 \neq 0$, we conclude that $A$ is invertible. And that its inverse is given by $$
A^{-1} = \frac{1}{-8}\begin{pmatrix}8 & -2 \\
-4 & 0\end{pmatrix} = \begin{pmatrix}-1 & 1 / 4 \\
1 / 2 & 0\end{pmatrix}.\quad\blacklozenge
$$
But what about in general when it is not a $2\times 2$ matrix? Let us think about this. Suppose we have an $n\times n$ matrix $A$ such that $AB = I_{n}$. Then recall the matrix-vector form of matrix multiplication:$$
\begin{align*}
AB = A[\vec b_{1} \ \vec b_{2}\ \cdots \ \vec b_{n}]=I_{n} \\
\implies [A\vec b_{1} \ A\vec b_{2}\ \cdots \ A\vec b_{n}]=[\vec e_{1} \ \vec e_{2}\cdots \vec e_{n}]
\end{align*}
$$where $\vec e_{k}$ are the columns of $I_{n}$ and $\vec b_{k}$ are the columns of $B$. So this tells us that the $k$-th column of $B$ solves the equation $$
A\vec b_{k} = \vec e_{k}
$$which we can solve by row reduction, $$
[A\ \vdots \ \vec e_{k}]
$$Now, if we can row reduce $A$ to a reduced-row echelon form (RREF), then $A$ would have become an identity matrix itself (as $A$ is square), in which case $$
[A\ \vdots \ \vec e_{k}] \stackrel{\text{row}}\sim [I_{n}\ \vdots \ \vec b_{k}].
$$
We can in fact do all the columns all at once. And we have the following proposition / algorithm:
> **Proposition.**
> Let $A$ be a square $n\times n$ matrix. Then $A$ is invertible if and only if $A \stackrel{\text{row}}\sim I_{n}$ and that $A^{-1}$ can be obtained by performing the row reduction $$
[A \ \vdots\ I_{n}] \stackrel{\text{row}}\sim [I_{n}\ \vdots \ A^{-1}].
$$
**Example.**
Decide if the following matrix $A$ is invertible or singular, and find its inverse $A^{-1}$ if exists: $$
A = \begin{pmatrix}4 & 1 & 1 \\
1 & 0 & 1 \\
2 & 3 & 1\end{pmatrix}.
$$
$\blacktriangleright$ Let us augment $A$ with the $3\times 3$ identity matrix $I_{3}$, and row reduce to RREF to see if the $A$ part becomes an identity matrix: $$
[A\ \vdots \ I_{3}]=\begin{pmatrix}4 & 1 & 1 & \vdots & 1 & 0 & 0 \\
1 & 0 & 1 & \vdots & 0 & 1 & 0 \\
2 & 3 & 1 & \vdots & 0 & 0 & 1\end{pmatrix} \stackrel{\text{row}}\sim
\begin{pmatrix}1 & 0 & 0 & \vdots & \frac{3}{8} & -\frac{1}{4} & -\frac{1}{8} \\
0 & 1 & 0 & \vdots & -\frac{1}{8} & -\frac{1}{4} & \frac{3}{8} \\
0 & 0 & 1 & \vdots & -\frac{3}{8} & \frac{5}{4} & \frac{1}{8}\end{pmatrix}
$$Since the left side turns into an identity matrix, we conclude that $A$ is indeed invertible, with $$
A^{-1} = \begin{pmatrix} \frac{3}{8} & -\frac{1}{4} & -\frac{1}{8} \\
-\frac{1}{8} & -\frac{1}{4} & \frac{3}{8} \\
-\frac{3}{8} & \frac{5}{4} & \frac{1}{8}\end{pmatrix}.\quad\blacklozenge
$$
## Elementary row matrix factorization.
Since a square $n\times n$ matrix $A$ is invertible if and only if it can be row-reduced to the identity matrix as its RREF, we recall by elementary row operations we can then factorize $A$ as $$
A = E_{1}^{-1}E_{2}^{-1}\cdots E_{k}^{-1} I_{n}= E_{1}^{-1}E_{2}^{-1}\cdots E_{k}^{-1},
$$where $E_{i}$ is the elementary row matrix associated with the $i$-th elementary row operation of taking $A$ to $I_{n}$, and that $E^{-1}_{i}$ is the elementary row matrix associated to the **reverse** (hence inverse) of the $i$-th elementary row operation. We can also write above as $$
E_{k}E_{k-1}\cdots E_{1} A = I_{n},
$$which shows $$
A^{-1} = E_{k}E_{k-1}\cdots E_{1}.
$$
In short, this shows that if $A$ is invertible, then it is a product of elementary row matrices, and vice versa.
## Pivots consideration.
In fact, if a square $n\times n$ matrix $A$ can be row reduced to the identity matrix $I_{n}$, it must have pivot every row and pivot every column. Indeed, if there is a column that has no pivot, then we cannot get to $I_{n}$ and vice versa. Using the rank terminology, we can rephrase this as
> A square $n\times n$ matrix $A$ is invertible if and only if $\text{rank}(A)=n$, if and only if an echelon form of $A$ has $n$ pivots.
**Example.**
Is the matrix $$
A = \begin{pmatrix}1 & 1 & 1 \\
1 & 2 & 2 \\
2 & 2 & 2\end{pmatrix}
$$invertible?
$\blacktriangleright$ A quick row reduction reveals that $$
A \stackrel{\text{row}}\sim \begin{pmatrix}1 & 1 & 1 \\
1 & 2 & 2 \\
0 & 0 & 0\end{pmatrix}
$$So we already see that rank of $A$ cannot be $3$, hence $A$ is not invertible. $\blacklozenge$
**Remark, but also caution.** As it turns out, elementary column operations on a matrix $A$ also preserves the rank, as well as elementary row operations. So one could use a mix of row and column operation **if the only goal is to determine rank**. Note, you cannot use column operations to compute the inverse of the matrix $A$ as above .... unless you augment an identity matrix on the bottom like this $\begin{pmatrix}A\\\cdots\\I_{n}\end{pmatrix} \stackrel{\text{col}}\sim \begin{pmatrix}I_{n}\\ \cdots \\A^{-1}\end{pmatrix}$, but must only use column operations. **Avoid using a mix** of row and column operations to find inverse of a matrix.
## Columns consideration.
We can actually say something about the columns of an invertible matrix.
> **Theorem.**
> Let $A$ be an $n\times n$ square matrix over $\mathbb{R}$. Then the following are equivalent.
> (1) $A$ is invertible.
> (2) The columns of $A$ are linearly independent.
> (3) The columns of $A$ span $\mathbb{R}^{n}$
**Remark.** This applies to fields (choices of scalars) other than $\mathbb{R}$ as well.
$\blacktriangleright$ Proof.
$(1\implies 2)$
Suppose $A$ is an $n\times n$ invertible matrix. Let $\vec a_{1},\vec a_{2},\ldots,\vec a_{n}$ be the columns of $A$. Consider the homogeneous equation $$
c_{1}\vec a_{1}+c_{2}\vec a_{2} + \cdots + c_{n}\vec a_{n} = \mathbf{0}.
$$By the matrix-vector formula, this can be re-written as $$
A \begin{pmatrix}c_{1}\\c_{2}\\\cdots \\c_{n}\end{pmatrix} = \mathbf{0}.
$$Since $A$ is invertible, $A^{-1}$ exists, and we can multiply both sides by $A^{-1}$, and get $$
A^{-1}A \begin{pmatrix}c_{1}\\c_{2}\\\cdots\\c_{n}\end{pmatrix} = A^{-1}\mathbf{0}\implies \begin{pmatrix}c_{1}\\c_{2}\\\cdots\\c_{n}\end{pmatrix}=\mathbf{0}.
$$So $c_{1}=c_{2}=\cdots =c_{n}=0$. Hence the columns of $A$ are linearly independent.
$(2 \implies 3)$
Suppose $A$ is an $n\times n$ matrix whose columns are linearly independent. We will show the columns of $A$ span $\mathbb{R}^{n}$, that is, $\text{columnspace}(A)=\mathbb{R}^{n}$. That is, if we write $\vec a_{1},\ldots,\vec a_{n}$ as the columns of $A$, we want to show $$
\operatorname{span} (\vec a_{1},\ldots,\vec a_{n})=\operatorname{span} (\vec e_{1},\ldots,\vec e_{n}).
$$
Now, the inclusion $\operatorname{span} (\vec a_{1},\ldots,\vec a_{n}) \subset \operatorname{span} (\vec e_{1},\ldots,\vec e_{n})=\mathbb{R}^{n}$ is clear, as the vectors $\vec a_{1},\ldots \vec a_{n}$ are all in $\mathbb{R}^{n}$.
Now we show each $\vec e_{k}$ is in $\operatorname{span} (\vec a_{1},\ldots,\vec a_{n})$. This amounts to showing $\vec e_{k}$ is a linear combination of $\vec a_{1},\ldots,\vec a_{n}$. And we show this by showing the augmented matrix $$
[\vec a_{1}\ \vec a_{2}\ \cdots \ \vec a_{n} \ \vdots \ \vec e_{k}]
$$is consistent. But since then $n$ vectors $\vec a_{1},\ldots,\vec a_{n}$ are linearly independent, the RREF has $n$ pivots in the first $n$ columns. So the system would always be consistent. Hence each $\vec e_{k}\in \operatorname{span}(\vec a_{1},\ldots \vec a_{n})$. So $\text{CS}(A)=\mathbb{R}^{n}$.
$(3\implies 1)$
Suppose the columns $\vec a_{1},\ldots,\vec a_{n}$ of $A$ span $\mathbb{R}^{n}$. Then we can write each $\vec e_{k}$ as a linear combination with $\vec a_{1},\ldots,\vec a_{n}$. So we can choose $n^{2}$ coefficients $c_{ij}$ such that $$
\begin{align*}
c_{11}\vec a_{1} + c_{21}\vec a_{2}+\cdots + c_{n1}\vec a_{n} & = \vec e_{1} \\
c_{12}\vec a_{1} + c_{22}\vec a_{2}+\cdots + c_{n2}\vec a_{n} & = \vec e_{2} \\
\vdots \\
c_{1n}\vec a_{1} + c_{2n}\vec a_{2}+\cdots + c_{nn}\vec a_{n} & = \vec e_{n} \\
\end{align*}
$$But the each equation is just a matrix-vector multiplication $$
A\begin{pmatrix}c_{11}\\c_{21}\\\vdots\\c_{n1}\end{pmatrix} = \vec e_{1},\ldots,A \begin{pmatrix}c_{1n} \\c_{2n}\\\vdots\\c_{nn}\end{pmatrix} = \vec e_{n},
$$which shows we have $$
AC = I_{n}
$$for some matrix $C$. Hence $A$ has a right-inverse $C$, which (if we believe right-inverse implies left-inverse), $A$ is invertible. $\blacksquare$
**Remark.** Since an $n\times n$ matrix $A$ is invertible if and only if the columns of $A$ are both linearly independent and form a spanning set for $\mathbb{R}^{n}$, this means they form a **basis** for $\mathbb{R}^{n}$.
## Some properties of inverses.
First we have a sock-and-shoe result:
> **Proposition.** If $A$ and $B$ are both $n\times n$ and both are invertible, then their product $AB$ is also invertible. In particular $$
(AB)^{-1}=B^{-1}A^{-1}.
$$
$\blacktriangleright$ Proof. Since $A,B$ both invertible, both $A^{-1}$ and $B^{-1}$ exist. We simply compute to show the proposed quantity $B^{-1}A^{-1}$ is the inverse to $AB$: $$
AB(B^{-1}A^{-1})=A(BB^{-1})A^{-1}=AI_{n}A^{-1}=AA^{-1}=I_{n},
$$and $$
(B^{-1}A^{-1})(AB)=B^{-1}(A^{-1}A)B=B^{-1}I_{n}B=B^{-1}B=I_{n}.
$$Hence we see the quantity $B^{-1}A^{-1}$ when multiplied to $AB$ on the left and on the right both gives $I_{n}$. Hence $(AB)^{-1}=B^{-1}A^{-1}$. $\blacksquare$
The reason we can do it column-wise only is because of the transpose respects the inverse:
>**Proposition.** If $A$ is invertible, then its transpose $A^{T}$ is also invertible. In particular $$
(A^{T})^{-1} = (A^{-1})^{T}.
$$
$\blacktriangleright$ Proof. Since $A$ is invertible, then $A^{-1}$ exists such that $AA^{-1}=I_{n}$ and $A^{-1}A=I_{n}$. Then taking transpose of both sides, and using the socks and shoes property of transpose of products, we get $$
(AA^{-1})^{T}=(I_{n})^{T} \implies (A^{-1})^{T}A^{T}=I_{n}
$$and $$
(A^{-1}A)^{T}=(I_{n})^{T} \implies A^{T}(A^{-1})^{T}=I_{n}.
$$So we see that $(A^{-1})^{T}$ when multiplied to $A^{T}$ on the left and right both yield $I_{n}$, so $(A^{-1})^{T}$ is the inverse of $A^{T}$. $\blacksquare$
Some more arithmetical properties of inverses, which I will leave to you as exercises.
> **Properties.**
> (1) If $A$ is invertible, then for any positive integer $k$ the matrix power $A^{k}$ is invertible, whose inverse is $(A^{k})^{-1}=(A^{-1})^{k}$.
> (2) If $A$ is invertible, and scalar $c\neq 0$, then $cA$ is also invertible, whose inverse is $(cA)^{-1}=\frac{1}{c}A^{-1}$.
Try your hand at proving those above! Hint. Compute the products of the matrix and its proposed inverse on the left and right, see both gives you identity matrix.
## Invertible matrix theorem.
It is good to collect a list of equivalent statements to when is a square matrix $A$ invertible. This is sometimes known collectively as invertible matrix theorem. We will add to this list later, but here is what we have discovered:
> **Invertible matrix theorem. First look.**
> Let $A$ be an $n\times n$ matrix, then the following are equivalent (TFAE):
> (1) $A$ is invertible, namely there exists a matrix $B$ such that $AB=I_{n}$ **and** $BA=I_{n}$.
> (2) $A$ is right-invertible, namely there exists a matrix $B$ such that $AB=I_{n}$.
> (3) $A$ is left-invertible, namely there exists a matrix $B$ such that $BA = I_{n}$,
> (4) $\det (A) \neq 0$ (we will investigate determinants more later).
> (5) $A$ can be row-reduced to the identity matrix $I_{n}$.
> (6) $\text{rref(A)} = I_{n}$.
> (7) $\text{rank}(A)=n$.
> (9) $A$ has pivot in every row.
> (9) $A$ has pivot in every column.
> (10) The columns of $A$ are linearly independent.
> (11) The columns of $A$ span $\mathbb{R}^{n}$.
> (12) The columns of $A$ form a basis for $\mathbb{R}^{n}$.
> (13) $A$ can be factored as a product of elementary matrices only.
> (14) $A^{T}$ is invertible.
> (15) The linear system $A\vec x = \vec b$ for any $\vec b \in \mathbb{R}^{n}$ has unique solution.
We will add more to this list as we learn more about matrices and their connection to linear transformations.